24 DEC 2013 by ideonexus
The Problem With Experimentation in the Real World
Government policies—from teaching methods in schools to prison sentencing to taxation —would also benefit from more use of controlled experiments. This is where many people start to get squeamish. To become the subject of an experiment in something as critical or controversial as our children’s education or the incarceration of criminals feels like an affront to our sense of fairness and our strongly held belief in the right to be treated exactly the same as everybody else. After all, i...
1 
Timo Hannay observes that we cannot experiment with classrooms and prisons because finding one experiment works means another didn't, creating winners and losers and offending our sense of justice.
02 JAN 2011 by ideonexus
A Classic Example of Selective Reporting
One of the classic examples of selective reporting concerns the testing of acupuncture in different countries. While acupuncture is widely accepted as a medical treatment in various Asian countries, its use is much more contested in the West. These cultural differences have profoundly influenced the results of clinical trials. Between 1966 and 1995, there were forty-seven studies of acupuncture in China, Taiwan, and Japan, and every single trial concluded that acupuncture was an effective tre...Studies of acupuncture in the West and East come up with very different assessments of it effectiveness.
01 JAN 2011 by ideonexus
Publication Bias Produces a "Decline Effect"
Jennions, similarly, argues that the decline effect is largely a product of publication bias, or the tendency of scientists and scientific journals to prefer positive data over null results, which is what happens when no effect is found. The bias was first identified by the statistician Theodore Sterling, in 1959, after he noticed that ninety-seven per cent of all published psychological studies with statistically significant data found the effect they were looking for. A “significant” re...Folksonomies: research decline effect
Folksonomies: research decline effect
Because publications are biased towards positive results, when a phenomenon produced in earlier studies turns out not to be true, then later studies will increasingly have difficulty reproducing the results.
